AGILEST Approach: Using Machine Learning Agents to Facilitate Kinesthetic Learning in STEM Education through Real-time Touchless Hand Interaction

There is an increasing interest in creating interactive learning applications using innovative interaction technologies, especially in STEM (Science, technology, engineering, and mathematics) subjects. Recent developments in machine learning have allowed for nearly perfect hand-tracking recognition, introducing a touchless modality for interaction within Augmented Reality(AR) environments. However, the research community has not explored the pedagogical approach of Kinesthetic Learning or “Learning by Doing”, hand tracking, and machine learning agents combined with Augmented Reality technology. Fundamentally, this exploration of touchless interaction technologies has taken on new importance in the new post-COVID world. Meanwhile, machine learning has gained attention for its ability to enhance personalized learning and play a vital new role as a virtual instructor. This paper proposes a novel approach called the AGILEST approach, which uses machine learning Agents to facilitate interactive kinesthetic learning in STEM education through touchless interaction. The first case study for this approach will be an AR learning application for chemistry. This application uses real-time touchless hand interaction for kinesthetic learning and uses a machine learning agent to act as both trainer and assessor of the user. The evaluation of this research has been conducted remotely through a usability study with expert reviewers, which includes 15 young researchers with peer-reviewed work in Human-Computer Interaction & AR and 2 subject experts STEM teachers at the secondary school level. The usability evaluation through NASA Task Load Index (NASA-TLX), Perceived Ease of Use(PUEU), and Perceived Usefulness(PU) with expert reviewers provide positive feedback about this approach for productive learning gain, engagement and interactiveness in learning STEM subjects.

Current Challenges and Future Research Directions in Augmented Reality for Education

The progression and adoption of innovative learning methodologies signify that a respective part of society is open to new technologies and ideas and thus is advancing. The latest innovation in teaching is the use of Augmented Reality (AR). Applications using this technology have been deployed successfully in STEM (Science, Technology, Engineering, and Mathematics) education for delivering the practical and creative parts of teaching. Since AR technology already has a large volume of published studies about education that reports advantages, limitations, effectiveness, and challenges, classifying these projects will allow for a review of the success in the different educational settings and discover current challenges and future research areas. Due to COVID-19, the landscape of technology-enhanced learning has shifted more toward blended learning, personalized learning spaces and user-centered approach with safety measures. The main findings of this paper include a review of the current literature, investigating the challenges, identifying future research areas, and finally, reporting on the development of two case studies that can highlight the first steps needed to address these research areas. The result of this research ultimately details the research gap required to facilitate real-time touchless hand interaction, kinesthetic learning, and machine learning agents with a remote learning pedagogy.

Potential security and privacy issues in zero UI touchless technology

Touchless technology often called Zero User Interface (UI) has begun to permeate every aspect of our lives as its use became necessary for hygiene measures in public places. The evolution of touchless technology replacing touchscreen interaction started as a luxury concept to give a fancier look to digital interactions, but now it has gained real value as a health-oriented interaction method. Switching to a touchless interface reduces common touchpoints, which help to safeguard against the spread of pathogens. Although the evolution of touchless technology is not new, its use massively increased due to its inherent hygienic nature during the COVID-19 pandemic. However, this investment in a new form of digital interaction has several privacy and security issues that need attention, in order to allow for safe human–machine interaction to cope with security breaches and cyber-attacks to protect our credentials. This paper outlines the potential security and privacy issues concerning Zero UI adoption in various technologies that need to be considered if one wishes to adopt responsible technology practices with this technology.

From luxury to necessity: Progress of touchless interaction technology.

Touchless Technology is facilitating the move to Zero User Interface(UI) propelled by the COVID-19 pandemic which has accelerated the use of this technology due to hygiene requirements. Zero UI can be defined as a controlled interface that enables user interaction with technology through voice, gestures, hand interaction, eye tracking, and biometrics such as facial recognition and contactless fingerprints. Smart devices, IoT sensors, smart appliances, smart TVs, smart assistants and consumer robotics are predominant examples of devices in which Zero UI is becoming increasingly adopted. These control interfaces include natural interaction modes such as voice or gestures. Touchscreens and shared devices such as kiosks, self-service counters and interactive displays are present in our everyday lives. Each of these interactions however is a concern for consumers in a post-COVID-19 world where hygiene is of utmost importance. The one-stop solution to hygienic interactions includes touchless technology such as voice control, remote mobile screen take over, biometric, and gesture control as Zero User interfaces. With the breakthroughs in image recognition and natural language processing, powered by advanced computer vision and machine learning, "Zero UI" is becoming a new normal. This paper is focusing on the progress of the touchless interaction technology during the COVID-19 pandemic, which actually accelerated development in this concept and moved it from being a luxury to a life necessity.

Covid-19 and Challenges for LearningTechnology Adoption in Pakistan

Creating educational disruption everywhere, the Covid-19 pandemic has hindered the lives of students and, sadly, will probably have a lasting impact on their future academic lives. What has gone relatively unnoticed is that it has created far more difficulties in developing countries. This is due to the fact that these countries were already lacking in Internet accessibility, e-learning solutions providers, and government policies for developing localized education technology. Students also lack personal resources. In managing the Covid-19 crisis better than many countries, Pakistan avoided the need for a full lockdown and set an example for the world. Its savvy policies even kept the economy running. Despite its proximity to neighbors China (where the first Covid-19 infection was found) and India (the second-most-affected country), Pakistan is surprisingly safe when compared with Europe and the U.S., with around a 98 percent recovery rate.